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ABSTRACT 


Certain  specular  types  of  sensor  images  (e.g.,  laser)  contain  vital  information  which  is  difficult  to  glean 
from  non-specular  sources.  The  present  and  increasing  deluge  of  these  types  of  images  has  created  a  critical 
need  for  image  processing  algorithms  which  reduce  the  workload  for  the  image  analyst  by  performing  some  of 
his/her  functions  automatical  ly.  Many  of  these  algorithms  are  based  on  image  segmentation — a  procedure 
having  1.)  high  separability  between  target  object  and  background  and  2.)  low  computational-intensity 
implementation  as  two  key  goals.  A  large  number  of  algorithms  for  automatic  segmentation  of  images  have 
been  tendered,  most  occuring  within  the  Mumford-Shah  paradigm  which  uses  approximation  error,  boundary 
length,  and  variance  as  weighted  terms  in  an  energy  functional.  The  new  idea  of  the  present  paper  is  to 
generalize  the  Mumford-Shah  variance  energy  so  that  it  directly  measures  the  relative  smoothness 
memberships  of  target  and  object  background.  This  is  especially  important  in  the  application  to  segmentation 
of  specular  types  of  images  which  tend  to  require  the  separation  of  subtle  grades  of  smoothness  and  the 
unraveling  of  delicate  smoothness  space  interpolations.  The  fast  wavelet  transform  answers  the  purposes  of 
efficient  determination  of  smoothness  membership  at  global  as  well  as  local  levels  and  works  well  in  a  quad 
tree  architecture. 
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1.0  FAST,  AUTOMATIC  IMAGE  SEGMENTATION 


In  sensors  employing  wavelengths  comparable  to  the  physical  dimensions  of  the  object  or  background  being 
imaged,  bright,  crest-on-crest  and  dark  trough-on-trough  interference  features,  called  speckle,  figure 
prominently.  In  most  instances,  the  presence  of  these  happenstance  reinforcements  and  cancellations 
complicates  the  extraction  of  application-important  objects  present  in  the  image.  Still,  certain  specular  types 
of  sensor  images  contain  vital  information  which  is  difficult  or  impossible  to  glean  from  the  non-specular 
imaging  systems.  Further,  the  greater  numbers  of  sensor  systems  being  deployed,  and  improvements  to  the 
resolution  of  these  systems,  have  created  a  deluge  of  images  for  image  analysts  to  cope  with.  Consequently, 
there  is  a  critical  need  for  image  processing  algorithms  which  reduce  the  workload  for  the  image  analyst  by 
performing  some  of  his/her  functions  automatically. 

An  example  automatic,  fast  algorithm,  based  on  segmentation  and  designed  to  assist  the  image  analyst, 
is  Northrop  Grumman  Corporation’s  Early  Vision  Image  Enhancement,  depicted  in  Figure  1.  The  image  is 
segmented  into  regions  differing  in  brightness  in  order  to  permit  the  localization  of  gray-level  assignments 
within  these.  This  provides  a  better  informational  channel  match  at  the  “gray-scale  assignment  channel”  so 
that  higher  rates  of  semantic  information  may  be  conveyed  to  the  viewer4. 

An  enormous  number  of  algorithms  for  automatic  image  segmentation  have  been  developed.  Most  of 
these  may  be  characterized  as  various  types  of  Mumford-Shah  segmentations5.  Typically,  the  split  and  merge 
operations  driven  by  the  Mumford-Shah  energies  are  performed  in  a  convenient  tree  algorithm  where 
operations  at  coarse,  global  levels  greatly  reduce  the  computational  intensity  for  the  fine,  local  levels.  The 
classic  quad  tree  segmentation  algorithm  is  an  important  benchmark  coarse-to-fine  architecture  for  performing 
Mumford-Shah  segmentation6. 

In  particular,  the  Mumford-Shah  model  seeks,  given  an  image  g(x),  a  piecewise  smoothed  image  u(x) 
with  a  set  K  of  abrupt  discontinuities,  the  edge  set  of  the  segmentation.  This  is  sought  by  minimizing  the 
functional 

E(u,K)  =  J  [|Vu(x)|2  +  (u-g)2]dx  +  length(K)  Eq.  1. 

£2\K 

Where  Q\K  is  the  image  domain  sans  boundary  set5. 


4. The  problem  of  conveying  semantic  information  across  a  communication  channel  is  the  “Level  B”  problem 
referred  to  in  The  Mathematical  Theory  of  Communication.  Shannon  and  Weaver,  University  of  Illinois  Press, 
1949. 

5. Jean-Michel  Morel  and  Sergio  Solimini,  Variational  Methods  in  Image  Segmentation.  Birkhauser,  1995.  ( 
See  also:  D.  Mumford  and  J.  Shah,  “Optimal  Approximations  by  Piecewise  Smooth  Functions  and  Associated 
Variational  Problems,”  Communications  on  Pure  and  Applied  Mathematics,  vol.  XLII  No.  4,  1989.) 

6.  Bart  M.  ter  Haar  Romeny  (Ed.),  Geometry-Driven  Diffusion  in  Computer  Vision.  Kluwer,  1994. 
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Figure  1.  Early  Vision  Image  Enhancement,  an  example  image  segmentation  application. 

An  ordinary  generalization  of  Eq.  1  expresses  the  Mumford-Shah  energy  as  the  s  um  of  three  terms — the 
variance  energy,  the  piecewise  approximation  error  energy,  and  the  edge  length  energy. 
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Eq.  2 


In  a  great  many  applications  (perhaps  most  applications)  the  second  term  is  zero  because  raw  data  compose  the 
segment  interiors  (apart  from  contrast  and  brightness  adjustments)  making  J  (u  -  g)2dx  =  0.  The  present 
paper  is  geared  toward  segmentations  of  this  type. 


2.0  ADAPTING  MUMFORD-SHAH  TO  THE  SPECULAR  PROBLEM 


Managing  the  relative  importance  of  length  energy  and  smoothness  energy  is  a  critical  issue  for  specular 
types  of  images.  This  manafests  as  a  tendency  for  the  segmentation  to  “shatter”  as  the  algorithm  proceeds 


from  large-scale  to  small-scale  split-and  -merge  operations.  An  example  shattered  segmentation  is  illustrated 
in  Figure  2.  The  stucco  wall  in  this  “pedestrian”  image  approximates  the  appearance  of  speckle.  The 
increasing  difficulty  in  separating  object  and  background  as  the  smaller  scale,  more-local  levels  are 
approached  (small-scale  shattering)  is  typical  of  specular  images. 


Figure  2.  An  image  segmentation  which  has  “shattered”  owing  to  the  presence  of  stucco  texture  (which  answers 
the  purpose  of  simulating  speckle  for  this  “pedestrian”  image). 


Toward  maintaining  greater  separability  between  object  and  background,  so  as  to  provide  against  or 
reduce  the  phenomenon  of  segmentation  shattering,  we  propose  a  slight  generalization  of  the  Mumford-Shah 
energy,  involving  some  recent  interpolation  space  ideas.  In  particular,  the  purpose  of  maintaining  high 
separability  between  object  and  background  is  more  directly  answered  by  replacing  the  variance  energy 
Evariance  with  a  “smoothness  separation”  energy  Erc|ative  separation  which  measures  the  relative  smoothness  space 
membership  of  object  and  background  during  segmentation: 
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Eq.  3. 
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In  particular,  we  propose  the  use  of  the  interpolation  junctor  for  the  interpolation  between  the  Besov  space  of 
the  object  of  interest  and  the  Besov  space  of  the  image  background  as  the  Erdat;ve  smoo,h„ess  tern1  for  the 
Mumford-Shah  segmentation. 

Besov  spaces  are  small  enough  (i.e.,  special  enough)  to  characterize  in  great  depth  much  of  what  is 
important  about  an  image.  For  example,  in  image  compression,  the  approximation  error  versus  the 
compressed  size  follows  a  power  law  given  by  the  Besov  membership  of  the  image7.  On  the  other  hand, 

Besov  spaces  are  large  enough  (i.e.,  general  enough)  to  hold  within  their  compass  the  great  bulk  of  image 
types  of  interest  to  applications.  Performing  segmentation  so  as  to  maintain  separate  smoothness  memberships 
locally  (in  terms  of  Besov  space  membership)  during  segmentation  extends  the  Mumford-Shah  theory  in  an 
important  way.  No  longer  limited  to  segmentation  into  “smooth”  versus  “coarse”  segments,  we  now  possess  a 
function-analytic  framework  for  pursuing  segmentations  between  regions  possessing  more  subtle  differences 
in  smoothness.  Further,  the  connections  to  the  interpolation  spaces  theory8  championed  by  Peetre  and  many 
others  allow  precise  statements  concerning  the  sense  in  which  a  segmentation  is  optimal. 

Owing  to  the  work  of  De  Vore  et  al7,  the  Besov  space  membership  may  be  estimated  locally  or  globally 
in  a  very  small  number  of  operations  using  orthonormal  wavelets.  Thus  the  separability  offered  by  the  relative 
Besov  space  memberships  comes  with  fast  algorithms  at  all  scales  during  a  segmentation.  The  parthenon 
image  along  with  its  decomposition  into  orthonormal,  Daubechies  wavelets  (via  the  Mallat  decomposition)  is 
presented  in  Figure  3.  The  LI  and  L2  energies  as  a  function  of  scale  are  also  shown. 

A  theorem  of  DeVore,  Jaweth,  and  Lucier  provides  that,  given  an  image  f  6  Lp(R2),  and  given  a 
multiresolution  analysis  Vj  with  wavelet  \| i,  in  order  for  f  to  belong  to  Besov  space  Bq“(Lp)  it  is  necessary  that: 
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Eq.  4. 


7.  Ronald  A.  DeVore,  Bjorn  Jawerth,  and  Bradley  J.  Lucier,  “Image  Compression  Through  Wavelet  Transform 
Coding,”  IEEE  Trans.  Info.  Theo.,  VOL.  38,  NO.2,  March  1992. 

8. J.  Bergh  and  J.  Lofstrom,  Interpolation  Spaces.  Springer- Verlag,  1976. 


Where  scale  is  2J  in  the  multiresolution  V,,  k  is  the  lattice  point  k  =  (k1«2J,  k2»2J),  k,,  k2  e  Z,  a  is  the  Besov 
“distributional  derivative,”  q  is  the  fine  smoothness  such  that  1/q  =  a/2  +  1/p,  and  where  also  f=Xkj 
Ckj,y\|/k,j  is  the  representation  in  the  orthonormal  wavelets  V|/k,j(p)  =  2JV)/(2Jp  -j)  in  wavelet  coefficients 

Ck.j.y  • 

Eq.  4  is  the  interpolation  functor  between  the  Besov  space  to  which  f  belongs  and  Lp.  The  wavelet  transform, 
offering  a  dyadic  architect  ure,  permits  the  estimation  of  the  Besov  space  functor  locally,  at  each  stage  of  a 
quad  tree  segmentation.  In  our  approach,  the  Erelative  sm0oth„ess  is  proportional  to  the  difference  between  the  K- 
functors  for  the  object  and  background  at  each  level  of  resolution.  Candidate  regions  tend  to  be  broken  off 
into  separate  segments  when  their  Besov  membership  differs  greatly  from  that  of  the  object  as  balanced 
against  the  impact  of  the  length  energy. 
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Figure  3.  An  image  and  its  decomposition  into  orthonormal  Daubechies  wavelets  (Mallat  decomposition)  are 
presented.  The  Lp  energy  of  the  wavelet  coefficients  as  a  function  of  scale  provides  an  estimate  of  the  Besov 
membership.  The  interpolation  K  functor,  a  measure  of  the  difference  in  smoothness  between  object  and 
background,  is  provided  locally  in  terms  of  wavelets  coefficients  as  the  segmentation  proceeds  from  coarse  to 
fine  segmentation. 


